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Abstract. Probabilistic systems are an important theme in AI domain. As the 
specification language, PCTL is the most frequently used logic for reasoning 
about probabilistic properties. In this paper, we present a natural and succinct 
probabilistic extension of /r-calculus, another prominent logic in the concurrency 
theory. We study the relationship with PCTL. Surprisingly, the expressiveness is 
highly orthogonal with PCTL. The proposed logic captures some useful prop¬ 
erties which cannot be expressed in PCTL. We investigate the model checking 
and satisfiability problem, and show that the model checking problem is in UP 
nco-UP, and the satisfiability checking can be decided via reducing into solving 
parity games. This is in contrast to PCTL as well, whose satisfiability checking is 
still an open problem. 


1 Introduction 

Temporal logics are heavily used in theoretical computer science and Al-related fields. 
Among those, modal /r-calculus receives a lot of attraction ever since Kozen’s seminal 
work Il20l . See for example, II2I19I31I3I . Moreover, various temporal logics including 
LTL Il26l , CTL IfT^ . CTL* lfT3l are extensively studied. It is known that their expres¬ 
siveness is strictly less IfTOl than /r-calculus (aka. juTL), and their model checking al¬ 
gorithm has been proposed: for CTL the problem can be solved in polynomial time, 
whereas for LTL the problem is PSPACE-complete Il29ll . 

Probabilistic systems, such as Markov chains and Markov decision processes, are 
an important theme in AI domain. To reason about properties for probabilistic systems, 
the logic CTL was first extended with probabilistic quantifiers in ifT^ , resulting in the 
logic PCTL. Intuitively, means that the probability of reaching f>-states along 

a-states is at least 0.9. At the same time, probabilistic LTL and its extension PCTL* 
have all been studied. As in the classical setting, model checking problem for PCTL 
can be solved in polynomial time, whereas only exponential algorithms are known for 
LTL 13 . There have also been several attempts to extend //TL with probabilities in the 
literature. As we shall discuss in the related work, the extensions are either highly non¬ 
trivial in terms of the complexity of the corresponding model checking and satisfiability 
problems, or hindered from the restriction of fixpoint nesting. 

We propose a natural and succinct extension of juTL in this paper, and name it PjuTL. 
The logic is acquired by equipping the next operator with probability quantifiers, and 
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keeping other parts as standard jiYh. We have for instance the formula vZ.(a A X-^-^Z). 
We investigate the model checking, expressiveness, and satisfiability problems of P/zTL. 

In detail, we first investigate the model checking problem of PpTL upon Markov 
chains. It turns out to be a straightforward adaptation of the classical algorithms for 
pTL, and the complexity remains in UP n co-UP. We then give a comprehensive study 
on the expressiveness of PyuTL by comparing with PCTL, and prove that PyuTL is or¬ 
thogonal with PCTL in expressiveness. However, for the qualitative fragments (i.e., 
probabilities may appear in a formula are only 0 and 1), we show that qualitative P/iTL 
is strictly more expressive (w.r.t. finite Markov chains). On the other side, the satisfi¬ 
ability checking is quite challenging; we exploit the notion of probabilistic alternating 
parity automata (PAPA, for short), and reduce the Satisfiability problem into the Empti¬ 
ness problem of PAPA. Further, this is reduced to solving parity games, and it is shown 
that both of these two problems are in 2EXPTIME. This is in contrast to PCTL as well, 
whose Satisfiability checking is still an open problem (cf. IM). 

An illustrating example We introduce a running example to motivate our work: Suppose 
there is a hacker trying to attack a remote server. The hacker has a supercomputer at 
hand and is trying to guess the password in a brute-force manner. For simplicity, we 
assume the password is a sequence of / letters, each of which is from ‘0’-‘9’, ‘a’-‘z’, 
and ‘A’-‘Z’. Therefore, the total number of possible passwords is n = 62^ The hacker 
let the supercomputer randomly generate a password, and see whether the decryption 
succeeds. If yes, the hacker wins; otherwise he tries with another one. However, if 
the supercomputer generates three wrong passwords in a row, it will be blocked for a 
certain amount of time until it can start another round of attacking — assuming that the 
password may be changed during the blocked moment, hence it does not make sense for 
the supercomputer to store all generated passwords. The whole process is illustrated in 
Fig- El Starting from si, we can see that the probability of eventually reaching attacked, 
i.e., the hacker decrypts successfully, equal 1, no matter how big / is (hence, the PCTL 
formula attacked holds), and we may conclude that the system is unsafe — this is of 
course against our intuition, as such system is considered to be safe if I is big enough. 
However, as we will show later, all PCTL formulae are not capable of expressing this 
property. By making use of P/iTL, such property of security can be characterized easily 
as follows: vZ.(-'attacked A X-^Z)) with p = where -^attacked denotes all other 

states in Fig. EJdifferent from 55 . 

Motivation from AI perspective The presented logic has the following potential appli¬ 
cation in AI domain: 

- First of all, Markov chains and Markov decision processes are the basic models 
in several areas of AI. As a logic with semantics defined w.r.t. such models, it 
could definitely be used in designating probability-relevant properties upon them. 
Particularly, the properties that could not be expressed by PCTL. 

- Motion planing is an important topic in AI area, where standard /iTL has once 
been adopted 0 , because of its powerful expressiveness and the decidability of 
its Satisfiability problem. Thus, we expect that P/iTL could be used in stochastic 
motion planning — since, PpTL is a decidability-preserving extension of /iTL. 
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Fig. 1. An illustration of the hacking process 

- Fixpoints play an important role in mathematics and computer science. In AI area, 
it is used to designate non-terminating behaviors of intelligent systems, such as 
maintenance goals ll28l . Fixpoints act as the elementary ingredients in P/rTL, hence 
such logic can also be used in such a situation. 

Related work Probabilistic extensions of juTL have been studied by many authors: e.g., 
/r-calculi proposed in 025117111121122l24ll interpret a formula as a function from states 
to real values in [0,1], whose semantics is different from P/rTL. A further extension of 
/i-calculus was proposed in 02^ . which is able to encode the full PCTL. However, the 
model checking and Satisfiability algorithms are still unknown for these calculi and are 
“far from trivial” UtM . The other probabilistic ju-calculus was introduced in lO along 
with a model checking algorithm for it. Moreover, it is able to encode PCTL formulae 
as well. However, that calculus only allows alternation-free formulae (cf. lfT4ll '). 

Very recently — and independently —, Castro, Kilmurray, and Piterman present 
another extension by adding hxpoints to full PCTL Q . The calculus they introduced is 
more expressive than logics PCTL and PCTL*. Moreover, it is also easy to see that it is 
a proper super logic of our logic P/rTL as well. They show the model checking problem 
is in NP nco-NP. We note that some examples in our paper are similarly investigated in 
Q. Since the logic in Q subsumes PCTL, its Satisfiability problem is also left open. 
However in this paper we show Satisfiability of P/rTL could be reduced to solving 
parity games, which makes this problem solvable in 2EXPTIME. 

2 Preliminaries 

In this paper, we fix a countable set Jl of atomic propositions, ranging over a,b,ai etc, 
and fix a countable set Z, of formula variables, ranging over Z, Zi etc. 

A Markov chain is a tuple M - {S,T,L), where 5 is a hnite set of states', T : 
S xS ^ [0,1] is the matrix of transition-probabilities, fulfilling Yjs'es ^ for 

every s e S', and L : S 2'^ is the labeling function. A pointed Markov chain is a parr 
(M, s) where M is a Markov chain iS, T, L) and s e S is the initial state. 

An (infinite )path n of M is an infinite sequence of states sq, si, • ■ such that s,- e S 
and T(si, s,+i) > 0 for each i. A basic cylinder cyl(so, si,...,««) of M is the set of 
inhnite paths having sq, s\,..., s„ as the prefix. 
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According to the standard theory of Markov process, the pointed Markov chain 
{M,s) uniquely derives a measure space {nM,s,Wob/^ where IIm.s consists of 
all infinite paths of M; Am,s is the minimal Borel field containing all basic cylinder of 
M (i.e., A(m,s) is closed under complementation and countable intersection); and the 
measuring function prob^ , fulfills: prob^ j(cyl(io, ii, ■ ■ ■, s„)) equals 0 if i sq, and 
equals ]”[;<« T{si, i,+i) otherwise. We say a set f* c 77^,^ is measurable if P e Am,s- II^ 
shows that the intersection of /7 m, j and an omega-regular set must be measurable. 

The syntax of PCTL formulae is described by the following abstract grammar: 

/ ::= T U I fl I I X-'’/I / A/I / V/I/U~V I/R~'’/ 

where ~e (>, >} and p e [0,1]. We also abbreviate tU~^/ and ±R~^/ as F~^/ and 
G~Pf, respectively. 

Semantics of a PCTL formula is given w.r.t. a Markov chain. For each PCTL for¬ 
mula / and a Markov chain M - (S,T, L), we will use [/Im to denote the subset of S 
satisfying /, inductively defined as follows. 

- |[t1m = 5;|[±1m = 0. 

- lajM -{s eS I a e L{s)}; [-.aIm - {s e S \ a i L(s)}. 

- IX't’/lM = e 5 I Y.s'em. ns, s') ~ p]. 

- Ifi ^ IiIm = I/i 1 m n |[/2 Im; I/i v /2IM = I/i 1 m u |[/2Im- 

- l[/iU~f/ 2 lM ^{seS \ prob^^TT e cyl(^) | tt N / 1 U/ 2 } ~ p] and = 

{seS \ prob^jTT e cyl(i) | tt N /i R/ 2 } ~ p}. 

In addition, for an infinite path tt = sq, ■si, ■ ■ ■ of M, the notation n |= / 1 U /2 stands for 
that there is some i > 0 such that s, e |[/ 2 ]]m and sj e |[/i]|m for each j < i. Meanwhile, 
n 1= / 1 R /2 holds if either n |= / 2 U(/i A / 2 ) or Sj e |[/ 2 ]]m for each J. To simplify 
notations, in what follows we denote by M, i |= / whenever s e I/Im holds. 

3 P/I XL, Syntax and Semantics 

In this section we present a simple probabilistic extension of modal /r-calculus, called 
P/tTL. The syntax of P/iTL formulae is depicted as follows: 

f ■■.^TUIal^alZIX-PflfAflfVflpZ.flyZ.f 

Semantics of a P/rTL formula is given w.r.t. a Markov chain M - (S,T,L) and an 
assignment e : Z ^ ^ ■ Similarly, for each P/rTL formula /, we denote by |[/]lM(e) 
the state set satisfying / under e. Inductively: 

- l[TlM(e) = S and = 0. 

- lajuie) = {i e 5 | a e L(i)) and 'i^ajuie) - {s e S \ a i L(i)}. 

- PlM(e) = e(Z). 

- [[X~'’/lM(e) = {seS\ T^s'emuie) ns, s') ~ p]. 

- Ifi A/2lM(e) = IfilMie) n UilMie) 
and If I V filmie) = Ifilmie) U IfilMie). 

- IpZ.flMie) = n{5' c 5 I inMielZ ^ 5']) c S'] and [vZ/ImC^) = [JiS' £ 5 | 
inM(e[Z^S'])^S']. 
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Indeed, ^Z.f^Mie) (resp. [[vZ./]|M(e)) could be computed as in the classical setting 
via the following iteration; 

1. let 5o = 0 (resp. 5o = 5); 

2. subsequently, let 5,+i = IfJuieYZ i-» 5,]); 

3. stops if S(+i =5 (, and returns S(. 

Note that the algorithm obtains a monotonic chain with such an iteration, and hence 
it must terminate within finite steps. Actually, ljuZ.fjMie) (resp. |[vZ./]]A/(e)) captures 
the least (resp. greatest) solution of Z = |[/lM(e[Z Z]) within 2^. 

Semantical definition of P/rTL formulae also yields the model checking algorithm. 

Theorem 1. The model checking problem of PpTL is in UP Hco-UP. 

Indeed, the proof is analogous to the non-probabilistic version II181321 and the only 
noteworthy difference lies from handling X~^- subformulae, opposing to □- and O- 
subformulae, which could be proceeded in (deterministic) polynomial time. 

In what follows, we directly denote by [/Im in the case that / is a closed formula 
(i.e., each variable of / is bound), and we also denote by M, i |= / if i e I/Im- 
Below we give some example properties: 

(1) The formula vZ.(a aX^^-^Z) describes that there exists an a-region, where each state 
has less than 0.2 probability to escape from it immediately (i.e., in one step). 

(2) vZ.(fl A X^^X^^’Z) says that there is a cycle in the Markov chain, such that a holds 
at least in every even step. 

(3) M, i 1= pZ.{a V X-° ®Z) if some a-state is reachable from s, but at each step, one just 
has some probability (not less than 0.6) to go on with the right direction. 

(4) The PpTL formula pZ.{b V (a A X-'Z)) holds if aUh holds along each path. It is 
stronger than the property described by the PCTL formula aU-'h. For the latter 
allows the existence of a-cycles. 

(5) As a more complicated example, the formula vZi.(a V//Z 2 .(a VX^‘’Z 2 ) aX-'Zi) just 
tells the story that “a will be surely encountered”, as described by F-'a with PCTL. 

Given a PpTL formula / and a bound variable Z, we use ^/(Z) to denote the sub¬ 
formula which binds Z in/. For example, let / = pZi(a AvZ 2 .(h aX^"‘’-^Z 2 ) vX^° ®Zi), 
then we have ^/(Zi) = / and ^/(Z 2 ) = vZ 2 .(b A X-®'^Z 2 ). 

We say that a P/rTL formula / is guarded, if the occurrence of each bound variable 
Z in ^f(Z) is in the scope of some X-operator. The following theorem could be proven 
in a same manner as that in 11311 . 

Theorem 2. For each PjiTL formula f, there is a guarded formula f such that ^f'^uie) 
Vf^mie) for every M and e. 

Thus, in what follows, we always assume that each PyuTL formula is guarded. 
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4 Expressiveness 

In this section, we will give a comparison between Pjt/TL and PCTL, and we are only 
concerned about closed PjuTL formulae. For a P/rTL formula / and a PCTL formula g, 
we say that / and g are equivalent if [/Im = for every Markov chain M, denoted 
^sf = g. 

First of all, we will show that some P/rTL formula could not be equivalently ex¬ 
pressed by any PCTL formula. 

Theorem 3. Let f - vZ.(a A X-°-^Z), then g ^ f for every PCTL formula g. 

Proof. To show this, we need hrst construct two families of Markov chains, namely. 
Mo, Ml,..., and M', MJ, M',.... 

For the hrst group, let M„ = {{sq, si,..., Sn},Tn,Ln), where: T„{so,so) = 1 and 
T„(si+i, Si) - 1 for each i < n (hence T„(si, sj) = 0 for any other s,, sj). In addition, 
Lniso) - 0 and L„{si) - [a] for each 0 < i < n. 

For the second ones, let M' = ({sg, Sj,..., s'}, T', L') where: T'„(s'„, s') = T'„(s'„, s^_j) 
0.5, T'(so. So) = 1, and 7''(sJ^j, s'.) - 1 for every / < n - 1. In addition, L'J.s'f) - 0 and 
L'lfs'f) - (a) for each 0 < i < n. 

Given a PCTL formula g, let N(g) be the maximal nesting depth of temporal- 
operators of g. According to IT] Thm. 10.45], we have that M', s' |= g if and only 
if M„, s„ 1= g whenever n > N(g). 

Observe the fact that M', s' |= / and M„, s„ ^ f for every n > 1. Assume that there 
exists some PCTL formula g fulhlling f = g, then we have 

MN{g), SN{g) N g <=^ Siv(^) 1 = / 

and hence it results in a contradiction. □ 

Conversely, the following theorem reveals that there also exists some PCTL formula 
that could not be equivalently expressed by any P/iTL formula. 

Theorem 4. Let f — then g ^ f for every (closed) PfiTL formula g. 

Proof Let M = ({si, S 2 , S 3 }, T, L) be the (family of) Markov chain(s) where: L(si) = 
L(s 2 ) = 0, L(s 3 ) = {a}, T(si,si) = ;ic,r(si,S 2 ) = y,r(si,S 3 ) = z, and r(s 2 ,S 2 ) = 
2 ’(s 3 , S 3 ) = 1, with x,y,z& (0,1) and x + y + z= 1. 

For every PCTL and/or closed PyuTL formula g, we let Px(g) be the proposition that 
“for the fixed x, there are infinitely many y making M, si |= g and there are infinitely 
many y making M, si ^ g”. We now show that if g is a closed P/rTL formula, then there 
exists some Xg < 1 such that Px(g) does not hold whenever x e (Xg, 1). 

- Such Xg can be arbitrarily chosen if g = ±, g - T, g = a or g - -la. 

- In the case that g - gi Ag 2 , assume by contradiction that such Xg does not exist, then 
it implies that for every x e ( 0 , 1 ), there exists some x' > x such that Pfig) holds. 
Observe that M, si |= g implies both M, si |= gi and M, si |= g 2 ', and M,si ^ g 
implies either M, si ^ gi or M, si ^ g 2 . Thus, we can infer that either x^, or Xg^ 
does not exist, which violates the induction hypothesis. 
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- Proof for the case of g V is similar to the above. 

- If .g = X~Pg' and p G (0,1), whenever x e (max{; 7 ,1 - p], 1), since ~e {>, >}, then 
M, Si 1= g iff M, Si 1= g' because y + z < p in such situation. In this case, we may 
just let Xg - maxjxgsp, 1 - p]. 

- If g = X-'g', then we need to distinguish two cases: 1) There exist e (0,1) 
such that M, Si |= g holds, then we can immediately infer that both M, S 2 1= g' and 
M, S 3 1= g'. In addition, observe that truth values of g' on S 2 and S 3 are irrelevant to 
X and y. It implies that in such case M, si |= g iff M, si |= g', and hence, we may just 
let Xg - Xg'. 2) There is no such x and y having M, si |= g holds, in such situation, 
Xg can be any number in ( 0 , 1 ). 

- If g = X^*^g', then the proof is similar to the above. 

- When g = X-®g' (or g = X^'g'), things would be trivial, because g could be reduced 
to T (resp. ±) in such case. 

- If g = pZ.g', we let go = ± and gi+i = g'[Z/gi]. Since that M is a 3-state Markov 
chain, then g and Vi <3 gi share the same truth value at every state of M. This indi¬ 
cates that all least fix-points could be eliminated w.r.t. such Markov chain. 

- When g = vZ.g', the preprocessing is almost similar, but we just replace g with 
Ai <3 gi where go = T. 

Now, for the PCTL formula / = such x/ does not exist, because, for every 

X e (0,1) we have: M,si |= / provided that y e [(1 - x)/2,1); and M,si ^ f if 
ye (0,(1- x)/2). This implies that Px(f) holds for every x e (0,1), and hence / cannot 
be equally expressed by any PyuTL formula. □ 

Note that the value 0.5 in the previous two theorems can be generalized to any other 
probability p e ( 0 , 1 ). 

We also provide a comparison on the qualitative fragments of PCTL and P/rTL. 
Probabilities occurring in such fragments can only be 0 or 1. 

Theorem 5. Every qualitative PCTL formula can be equally expressed by a qualitative 
PpTL formula. 

Proof. We will give a constructive translation procedure, which takes a qualitative 
PCTL formula g and outputs an equivalent qualitative P/rTL formula g. Inductively: 

1 . g = ± if g = ±, or its root operator is X^', or R^'; 

g = T if g = T, or its root operator is X-®, U-° or R-®. 

2- g = gi A_g^ if g = gi A g 2 ; and g = V gi if g = gi V g 2 . 

3- ? = X>V if ^ and g = X-'g' if g = X-'g'. 

4. g = pZ.{g2 V (gi A X^^Z)) if g = giU>''g 2 ; 

and g = vZ.(g 2 A (gi^^-'Z)) if g = giR-'g 2 . 

5. g - vZ.(g 2 V (gj A F> 0 g 2 A X^'Z)) = vZ.(g 2 V (gi A pZ'.{g 2 V X>''Z') A X^'Z)) if 

.g = .giU-'g 2 ; _ 

and g = pZ.{g 2 A (gj V G^'g 2 V X><'Z)) = pZ.{g 2 A (gj V vZ'.(g 2 A X^'Z') V X>''Z)) 

if.g = 


The proof of equivalence could be done by induction on the structure of the formula. □ 
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Note that Thm.|5] holds because we are only concerned about finite models in this 
paper. Interested readers may show that it is not true for infinite Markov chains. 

Theorem 6. The qualitative P^TL formula f — vZ.{a A cannot be expressed 

in qualitative PCTL. 


Proof. Construct a series of Markov chains such that each M” is the 

Markov chain ({s",s",..., 4'}, where r"(s", s") = 1 and r"(s" j, s") = 1 

for each i < n. In addition, L"{s") - {a} for each i + 1, and L"(i") = 0. 

For a given PCTL formula g, let g be the LTL formula obtained from g by dis¬ 
carding all probability quantifiers, e.g., we have g = a[J{b V G-ia) if g = V 

G>o. 6_,^) from s” the Markov chain M" has exactly one infinite path 7r„ = 

s", i", (Sq)^^, then for each n > 2 we have M", s” N g if and only if 7r„ |= g. It is 
shown in ||^ that M", s” N g iff ■^”+p ■s”+i N g in the case of « > A^'(g) = N'(g), 
where N'(g) and N'(g) are the nesting depth of X-operator of g and g, respectively. 
Thus, we have M'f,s” |= g iff N g in such situation. This implies that 

vZ.(a A X^^X^^Z) has no equivalent qualitative PCTL expression, because we cannot 
simultaneously have M", s” |= / and N / for each n >2. □ 


Note that the conclusion of Thm.|6]is also pointed out in fSl, and we here provide a 
detailed proof. Indeed, this proof also works for general PCTL formulae, and hence the 
property vZ.(a A even cannot be expressed by any PCTL formula. 


5 Automata Characterization 

In this section, we will define a new type of automata recognizing (pointed) Markov 
chains, called probabilistic alternating parity automata (PAPA, for short), and such 
automata could be viewed as the probabilistic extension of those defined in fiZX . 

A PAPA A is a tuple {Q, qo, d, Q) where; 2 is a finite set of states, q^ e Q is the 
initial state, d is the transition function to be defined later, and L? ; Q N, is a partial 
function of coloring-, in what follows, we say a state is colored if Q is defined for the 
state. 

The notion of transition conditions over Q is inductively defined as follows: 

1. ± and T are transition conditions over Q. 

2. For every a e Jl, the literals a and -la are transition conditions over Q. 

3. If q e Q, then g' is a transition condition over Q. 

4. If q e Q and p e [0,1], then 0~'’q is a transition condition over Q, where ~e {>, >). 

5. If qi,q 2 ^ Q then both qi V q 2 and qi A q '2 are transition conditions over Q. 

The transition function 6 assigns each state q e Qa transition condition over Q. 

We denote by Ra the derived graph of A, its vertex set is just Q, and there is an edge 
from q\ to q 2 iff q 2 appears in 6{qi). We say that A is well-structured, if for every path 
q\,q 2 , ■ ■ ■ ,qn that forms a cycle (i.e., q\ - q„) in Ra, we have that: 1) there exists some 
I < i < n such that diqi) = 0~^qi+i with some p e [0,1]; 2) there exists some 1 < j < n 
such that qj is colored. In what follows, we are only concerned about well-structured 
PAPA. 
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Given a pointed Markov chain (M, sq) with M - (S,T,L) and sq e 5, a run of A 
over (M, sq) is a Q x 5 -labeled tree (T, A) fulfilling: /l(vo) = ■so) for the root vertex 
Vo; and for each internal vertex v of T with /l(v) = (q, s) we require that 

- 6(q) + ±, and if b(cj) - T then v has no child; 

- a e L{s) if b(q) = a, and a i L{s) if 6{q) - -lo; 

- if 6{q) - q\ Aq 2 then v has two children vi and V 2 respectively having /i(vi) = (qi, s) 
and A{v 2 ) = (qi, s); 

- if 6(q) - qi V qi then v has one child v' with A(v') 6 {(o’!, s), {q 2 , i)}; 

- V has one child v' having A{v') = (q', s), if d(q) = 

- if Siq) - 0~^q’ then v has a set of children vi,..., v„ such that /l(v,) = {q’, si), 

where ~ P- 

For an infinite branch r = vq, vi,... of F, let be the number 

max{ n \ there are infinitely many i s.t. f3(proj[(/l(v,))) = n] 

where proj i(q,s) = q. A run ( T, A) is accepting if is an even number, for every infinite 
branch r of F. A pointed Markov chain (M, sq) is accepted by A if A has an accepting 
run over it. We denote by ^(A) the set consisting of pointed Markov chains accepted 
by A. 

Theorem 7. Given a closed PpTL formula f, there is a PAPA Af such that: M, i |= / 
iff{M, s) e A f), for each pointed Markov chain {M, s). 

Proof We just let Af = (Qf,qf,df,Qf), where: 

- Qf - {qg\ g ^ subformula of /}, and hence qf e Qf, 

- 6f is defined as follows: 

• SfiqAi) = ± and Sfiq^) = T; 

• dfiqa) - a and bfiq^a) - “'o; 

• bf{qg^/^gf) — qg^ A qg^ and bf{qg^\/gf) — qg^ V qgp, 

• Sfiqx-i’g) = 0~^qg-, 

• bf{qpiZ.g) — qg and bf{qvz.g) — qg^ 

• df(qz) - q@f{z)- 

- Qf is defined at every state qz with Z e X fulfilling: If Z is a //-variable (resp. 

V-variable), then Qf{qz) is the minimal odd (resp. even) number which is greater 
than every Qf(qz') such that is a subformula of ^/(Z). 

It could be directly examined that A/ is well-structured since / is guarded. The proof 
of equivalence can be similarly done as that in ll^ — the only different induction step 
is to deal with transitions being of 0~^’q (in that paper, the corresponding cases are 
aq and <>q). Actually, we can see that if a PAPA [Q, q, b, Q) corresponds to the P/(TL 
formula g, then the PAPA {Q U {q'], q', b[q' 0~^q], Gt) must correspond to □ 
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6 Satisfiability Decision 

It is known from Section|5]that the Satisfiability problem of P/rTL could be reduced to 
the Emptiness problem of PAPA. In this section, we will further reduce it to parity game 
solving. 

A parity game G is a tuple (V, E, C), where: V is a finite set of locations, and V 
could be partitioned into two disjoint sets and P'; £ c y x E is the set of moves, 
required to be total; and C : V N is a partial function of coloring, and we say a 
location v is colored, if C(v) is defined. In addition, for the game G, we require that 
each loop involves at least one colored location. 

Two players — player 0 and player 1, are respectively in charge of and V* when 
G is being played. A play of G starting from vq e V is an infinite sequence of locations 
Vo, vi,... made by player 0 and player 1 — for every i e N, the location v,+i is chosen 
by player 0 (resp. player 1) with (v,, v,+i) e E whenever v,- e (resp. v, e V'). 

Player 0 (resp. player 1) wins the play vo, vi,... if the maximal color occurring 
infinitely often in it is even (resp. odd) — and we say that a color c occurs in this play 
if there is some v, with C(v,) = c. 

A winning strategy for player i is a mapping Eli : P* ■ V' —> P, such that for 
every play vo, vi,..., player i always wins if Vj+\ - Hiivo,..., vj) whenever v/ e P'. In 
addition, //, is memoryless if Hi{vo ,..., v^) agrees with Hiivj) for every j. 

Theorems ( II15I34I18I L For a parity game G, from every location, there is exactly 
one player having a winning strategy. The problem of deciding the winner at a location 
is in UP OcO“UP. In addition, if a player has a winning strategy then she also has a 
memoryless one from the same location. 

We use Wi(G) to denote the set consisting of all locations from which player i has a 
winning strategy. 

Given a PAPA A - (Q, q, 6, Q), a gadget D of A is a finite directed acyclic digram 
{P, y) where P Q Q, y Q P x P, and for each q e P: 

1. if 6(q) - q', then q' e P and (q, q') e y; 

2. if 6{q) - qi hqi then q\,q 2 & P, and (q,qi),(q,q 2 ) e y, 

3. if 6(q) - qi V q 2 then there is some i e {1,2} such that qi e P and {q, q,) e y, 

4. q has no successor for the other cases. 

For convenience, we sometimes directly write q e D whenever D - (P, y) and q e P. 
We denote by 2)(A) the set consisting of all gadgets of A. Since we require that each 
PAPA A is well-structured, then D(A) must be a finite set. 

Given a sequence of gadgets D\,D 2 ,... such that D, — {Pi,yi), an infinite path 
within it is a sequence of states qi^i,..., q\y^,q 2 ,\, ■ ■ ■, <l 2 ,ti, ■ ■ ■ such that each {qiq, qi,j+\) e 
y, and diq^f - for some p, e [0,1]. We say such an infinite path is even 

(resp. odd) if the maximal color (w.r.t. Q) occurring infinitely often is even (resp. odd). 

We say that a gadget D - {P,y) is incompatible if there exist qi,q 2 e P and 6(qi) - 
a, S(q 2 ) = “'fl for some a e or there is some q e P with 6{q) = ±. Otherwise, we say 
that D is compatible. 

Let D be a gadget and E - {Di,..., D^} be a set of gadgets, we denote hy E \\- D 
if there exist k positive numbers x\,.. .,Xk such that: xi < 1, and for each q e D 
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with 6{q) — we have Yjq'eOi^i ~ P- We in what follows call xi,...,Xk the 

enabling condition. Note that the relation ih could be decided by solving a linear system 
of inequality. 

According to automata theory, we may construct a deterministic (word) parity au¬ 
tomaton A - (Q,^, 6, Q) were 6 : Qx !D(A) Q and is a total coloring function. It 
takes a gadget sequence as input, and accepts it if every gadget in it is compatible and 
every infinite path within it is even. 

Then, we may create a parity game Ga - (Va, Ea, Ca) for the PAPA A, in detail: 


- Va^V^U V\, where V° = and V\ = D(A) x Q. 

- Ea^ {({(Di,^i), ... ,{Dk,qk)},{E)i,qi)) | 1 < / < k) U 

{((D, {{Duqj),-.-,{Dk,qk)])\{Du...,Dk) Ih D, 

and each qi = 6(q, D,)]. 

- Ca{D,^ - Q(^, hence every location in V\ is colored. 

Theorem 9. Let the PAPA A — (Q, q, 6, Q), then J§f(A) + 0 if and only if there is some 
D e D(A) with q e D such that {(D, 6(q, D))} e WiffGAd- 

Proof. =>) Suppose that there is some pointed Markov chain (M = (S,T,L),s) e 
.Sf(A), then there exists some accepting run (T, T) of A on (M, s). 

We say a vertex v of T is a modal vertex if 5(proj j (/l(v))) is of the form 0~^q'. 
We denote by ||v|| the modal depth of v, i.e., the number of modal vertices among the 
ancestors of v. 

From each vertex v of T, we may obtain a set of vertices, denoted as cls(v), which 
involves v and all its descendants with the same modal depth. Since A is well-structured, 
then cls(v) must be a finite set. We also lift the notation by defining els V = Urev cls(v) 
for a finite vertex set V. 

In addition, each finite vertex set V of T derives a gadget D(y) = (Py, yv), where 
Py = {proj[(/l(v)) I V e y}, and iqi,q 2 ) e Jv if there are two vertices vi, V 2 e V, such 
that proj[(/l(v,)) = qi for i — 1,2 and V 2 is a child of vi. 

Let Vo be the root vertex of T, then we have /l(vo) = (q, s). We now let D - Dq - 
D(cls(vo)), then for each play Aq, (Do,^),/1i, {Di,qi),A 2 ,... with Aq - {D,6(q,D)) 
and each D, = (P,, y,), player 0 can control it and make the play to fulfill the following 
property: 

(*) For each i, there exists a finite set of vertices y having the same modal depth i, 
and there exists a state s, of M; and q' e P, iff there is some v^- e y such that 
d(vq') - (q', Si). In addition, (qi,q 2 ) e Ji iff Vq^ is a child of Vq^. 

For i - 0, we have Vq = cls(vo) and sq = s. Assume that (*) holds at step i, then player 
0 chooses the next location guided by the run as following: First, let VI be all modal 
vertices among y, and let V” be the set consisting of children of vertices in V'. Then, 
y" can be partitioned into several sets V'f ,..., y"j, according to the second component 
(assume proj 2 (/l(v')) = Sij for v' e V''j) labeled on the vertices. Player 0 then chooses 
the set {{Dki,qki),... ,(Dkk,^k)] as the next location, where = D(cls(y"p) and 

tji.j — d{qi^ Dij). 
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Then, according to the construction, for each {Dij, 6{qij)) we have some state Sij 
and the vertex set cls(T"p making property (*) holds, no matter how player 1 chooses. 
Let xi - T(si, Six), ■ ■ ■,^k - T(si, si^), we definitely have ^ 1 we also have 

Tjq"£Dij ~ P for sach q' e Dt such that 6(q') = 0~'’q” because {T, A) is an accepting 
run. Therefore, (Z),..., Di k) IH Di holds. 

We assert that each Dt - (qi,..qi) must be compatible — since {T, T) is accept¬ 
ing, no such q' e Di having 6{q') = ±, and if there exist q\,q 2 e Di with d(q'i) = a and 
5(^2) = ~'a, then we will both have a e L{si) and a i L(si). Also note that each infinite 
path within Do, D\,... corresponds to the first component of the labelings of an infinite 
branch of T, hence it must be even. According to A, we then conclude that this strategy 
is winning for player 0 form {(£> 0 ,5(^)}. 

<=) Let Ho be the (memoryless) winning strategy of player 0 from {{D,6(^)], 
where D is some gadget involving q. We say that a location / = {D^, q') e V\ is feasible 
if I may appear in some play under control of player 0 according to Hq. We create a 
Markov chain M - {S ,T, L) as follows. 

- First, let S = {i; | 1 is a feasible location) U {i'}. 

- Second, since each feasible location must be compatiable, then we may let £(s/) = 
{a e I there is some q' in Df. Meanwhile, we let L{s') - 0. 

- The transition matrix T is determined as follows: For each feasible location /, sup¬ 
pose that Hq{ 1) - {l\ - (D^' ,q^'),.. .,lk - (D^, q‘‘‘)], since {D‘' Ih D‘ then 

we have a set of enabling condition x\,... ,Xk. We let T{si, si.) - xj for each j, let 
T(si, s') ^ I- Xj, and let T(s', i') = L 

What left is to show that (M, si„) e -Sf(A), where lo is just (D, 6(q)). For each gadget 
D' such that / is feasible, we could obtain a forest (£/, Af), and in which each vertex q' 
is labeled with (q', si). Then from (which is an exact tree with (q, siJ labeled in the 
root), with a top-down manner, we connect the so far added tree T ; with every Ti> such 
that r e Hod) — i-e., for each q' in £; with 5{q') - 0~'’q", we add the vertex q" in Tf 
as a child — it can be seen that it must be the case that some edges connecting some 
leaves of £/ and the root(s) of T//. We denote the labeled tree finally get as (T, A), and it 
is indeed be an accepting run of A over (M, si^). □ 

Intuitively, player 0 could extract a winning strategy from an accepting run of A 
over any pointed Markov chain; and conversely, one can construct a pointed Markov 
chain accepted by A according to the (memoryless) winning strategy of player 0. 

As a consequence of Thm. |7] Thm. |8] and Thm. |9] we have the following main 
conclusion of this section. 

Theorem 10. Both the Emptiness problem of PAPA and the Satisfiability problem of 
PpTL are decidable, and both of them are in 2EXPTIME. 

Indeed, from Thm. |7] one can get a PAPA whose scale is linear in the size of the 
input formula, and an n-state PAPA could be converted to a parity game with scale 
2^*"’. From standard game theory (see II181321 . and see Il27l for an improved bound), 
and with a similar analysis of ll^ (see also the analysis of the coloring number in that 
paper), one can infer that this problem is in 2EXPTIME. 
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7 Discussion 

In this paper, we present the logic PpTL, a simple and succinct probabilistic extension 
of pTL. We have compared the expressiveness of these two kinds of logics; In gen¬ 
eral, PjuTL captures ‘local’ and ‘stepwise’ probabilities; whereas PCTL could describe 
‘global’ probabilities in the system. Hence, these two logics are orthogonal and com¬ 
plementary, and one can obtain a more powerful and expressive logic by combing them 
together, as done in Q. i.e., we may use formulae like (juZ.(a V X-°-®Z))U-°®(vZ'.(l7 A 
F>o.3z')) Model checking algorithm of such an extension can be acquired from those 
of the underlying logics. 

In this paper, we have also investigated the decision problem of PpTL, the key issue 
and the most challenging part is to deal with probabilistic quantifiers when doing re¬ 
duction to parity games, which is a highly nontrivial extension of the non-probabilistic 
case. As a cost, we have only now got an algorithm with double-exponential time com¬ 
plexity for solving it — in contrast, the Satisfiability problem for the standard pTL is 
in EXPTIME. 

Acknowledgement 

First and foremost, the authors would thank all the anonymous reviewers for the valu¬ 
able and helpful comments on this paper. We would also thank Nir Piterman for his 
valuable comments on our work. 

Wanwei Liu is supported by National Natural Science Foundation of China (Grant 
Nos. 61103012, 61379054 and 61272335). Lei Song is supported by Australian Re¬ 
search Council under Grant DP130102764. Ji Wang is supported by National Natural 
Science Foundation of China (Grant No. 61120106006). Lijun Zhang (corresponding 
author) is supported by National Natural Science Foundation of China (Grant Nos. 
61428208, 61472473 and 61361136002), the CAS/SAFEA International Partnership 
Program for Creative Research Teams. 


References 

1. C. Baier and J.-P. Katoen. Principles of Model Checking. MIT Press, 2008. 

2. B. Banieqbal and H. Barringer. Temporal logic with fixed points. In B. Banieqbal, H. 
Barringer, and A. Pnueli, editors, Temporal Logic in Specification, volume 398 of Lecture 
Notes in Computer Science, pages 62-74. Springer-Verlag, 1987. 

3. S. Berezin. Model Checking and Theorem Proving: A Unified Framework. Phd thesis, 
Carnegie Mellon University, Pittsburgh, PA, USA, Jan. 2002. 

4. N. Bertrand, J. Feamley, and S. Schewe. Bounded satisfiability for PCTL. In CSL, volume 16 
of LIPIcs, pages 92-106. Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, 2012. 

5. A. Bhatia, M. R. Maly, L. E. Kavraki, and M. Y. Vardi. Motion planning with complex goals. 
Robotics &- Automation Magazine, IEEE, 18(3):55-64, 2011. 

6. T. Brazdil, V. Forejt, J. Kretmsky, and A. Kucera. 

7. P. Castro, C. Kilmurray, and N. Piterman. Tractable probabilistic /r-calculus that expresses 
probabilistic temporal logics. In 32nd Symposium on Theoretical Aspects of Computer Sci¬ 
ence, volume 30 of Leibniz International Proceedings in Informatics, pages 211-223, 2015. 


14 


Wanwei Liu, Lei Song, Ji Wang, and Lijun Zhang 


8. R. Cleaveland, S. Iyer, and M. Narasimha. Probabilistic temporal logics via the modal /r- 
calculus. Theor. Comput. ScL, 342(2-3):316-350, 2005. 

9. J.-M. Couvreur, N. Saheb, and G. Sutre. An optimal automata approach to LTL model 
checking of probabilistic systems. In LPAR 2003, volume 2850 of Lecture Notes in Computer 
Science, pages 361-375. Springer, 2003. 

10. M. Dam. Translating CTL* into the modal yu-calculus. Technical Report ECS-LFCS-90-123, 
Laboratory for Foundations of Computer Science, University of Edinburgh, November 1995. 

11. L. de Alfaro and R. Majumdar. Quantitative solution of omega-regular games. In STOC, 
pages 675-683. ACM, 2001. 

12. E. Emerson and E. Clarke. Characterizing correctness properties of parallel programs using 
fixpoints. In Proc. of the 7th Int. Colloquium on Automata, Languages and Programming 
(ICALP’80), volume 85 of Lecture Notes in Computer Science, pages 169-181. Springer- 
Verlag, 1980. 

13. E. A. Emerson and J. Y. Halpern. Decision procedures and expressiveness in the temporal 
logics of branching time. Journal of the ACM, 33(1):151-178, 1986. 

14. E. A. Emerson and C. L. Lei. Efficient model checking in fragments of the propositional 
mu-calculus. In First IEEE Symposium on Logic in Computer Science, pages 267-278. Los 
Alamitos: IEEE Computer Society, 1986. 

15. Y. Gurevich and L. Harrington. Trees, automata, and games. \n Proceeding of 14th ACM 
Symposium on the Theory of Computing, pages 60-65, San Francisco, California, 1982. 

16. H. Hansson and B. Jonsson. A logic for reasoning about time and reliability. FAC, 6(5):512- 
535, 1994. 

17. M. Huth and M. Z. Kwiatkowska. Quantitative analysis and model checking. In LICS, pages 
111-122. IEEE Computer Society, 1997. 

18. M. Jurdzihski. Deciding the winner in parity games is in UPnco-UP. Information Processing 
Letters, 68(3): 119-124, 1998. 

19. J.-P. Katoen. Concepts, Algorithms, and Tools for Model Checking. FAU, Lehrstuhl fiir 
Informatik VII Friedrich-Alexander Universitat Erlangen-Niirnberg, 2 edition, 1998. Lecture 
Notes of the Course “Mechanised Validation of Parallel Systems”. 

20. D. Kozen. Results on the propositional yu-calculus. Theoretical Computer Science, 21 
354, 1983. 

21. A. Mclver and C. Morgan. Games, probability and the quantitative yu-calculus qmp. In LPAR, 
volume 2514 of Lecture Notes in Computer Science, pages 292-310. Springer, 2002. 

22. A. Mclver and C. Morgan. Results on the quantitative yu-calculus qmp. TOCL, 8(1):3, 2007. 

23. M. Mio. Game semantics for probabilistic modal p-calculi. PhD thesis. The University of 
Edinburgh, 2012. 

24. M. Mio. Probabilistic modal /r-calculus with independent product. Logical Methods in 
Computer Science, 8(4), 2012. 

25. C. Morgan and A. Mclver. A probabilistic temporal calculus based on expectations. In 
L. Groves and S. Reeves, editors, Proc. Formal Methods Pacific, pages 4-22. Springer, 1997. 

26. A. Pnueli. The temporal logic of programs. In Proc. of 18th IEEE Symposium on Foundation 
of Computer Science (FOCS’ 77), pages 46-57. IEEE Computer Society, 1977. 

27. S. Schewe. Synthesis of Distributed Systems. Phd thesis, Saarbriicken, 2008. 

28. M. P. Singh. Applying the mu-calculus in planning and reasoning about action. J. Log. 
Comput., 8(3):425^45, 1998. 

29. A. P. Sistla and E. M. Clarke. The complexity of propositional linear temporal logics. Journal 
of Assoc. Comput. Mach., 32(3):733-749, 1985. 

30. M. Y. Vardi. Automatic verification of probabilistic concurrent finite-state programs. In 
FOCS, pages 327-338. IEEE Computer Society, 1985. 

31. 1. Walukiewicz. Completeness of Kozen’s axiomatization of the propositional yu-calculus. 
Information and Computation, 157:142-182, 2000. 



A Simple Probabilistic Extension of Modal Mu-calculus 


15 


32. T. Wilke. Alternating tree automata, parity games, and modal //-calculus. Bull, Belg, Math, 
Soc, 8(2):359-391, 2002. 

33. P. Wolper. Temporal logic can be more expressive. Information and Control, 56(l-2):72-99, 
1983. 

34. W. Zielonka. Infinite games on finitely coloured graphs with applications to automata on 
infinite trees. Theoretical Computer Science, 200:135-183, 1998. 



